Performance Evaluation of Scene Registration and Stereo Matching for Artographic Feature Extraction

One goal of automated cartography is to generate an accurate 3-D model of man-made structures and natural terrain. Some of the most challenging problems in cartographic feature extraction occur in dense urban areas where the level of detail and scene clutter greatly complicate traditional map compilation techniques. In this paper, we describe experiments toward a comprehensive stereo analysis system to recover the 3-D description of an urban area using high-resolution aerial imagery. Given an area of interest in terms of geographic coverage, our system can automatically find the appropriate stereo pair using a spatial database, select control points to register the two images so that epipolar geometry is satisfied, and recover disparity information using two complementary matching techniques. In our research, we do not assume that the initial input images satisfy the epipolar geometry constraint because this is rarely the case in unrectified aerial imagery. Therefore, we argue that stereo mapping research must explicitly address error and uncertainty in both scene registration and stereo matching and that we need techniques to evaluate such errors in a rigorous manner. We also argue that in order to achieve robust behavior, multiple methods for scene feature extraction should be utilized, and if possible, their results should be integrated into a consistent framework. We describe techniques for scene registration using five different features that can be automatically extracted to provide control points for fine image registration. In the stereo matching process, two techniques are utilized: an area-based and a feature-based stereo matcher to generate a disparity map for a scene. We also present some preliminary results on a technique to merge the results of the stereo matching algorithms to provide improved information regarding height estimates. Finally, we describe techniques to generate rigorous performance analysis metrics to compare stereo matching algorithms based on a manually derived 3-D ground truth segmentation. The analysis includes the error estimation metrics for both height and delineation accuracy based on the measurements of deviations from manual estimates. These estimates are computed globally over the entire scene and locally on a structure-by-structure basis. Manuscript received October 1, 1990; revised May 6, 1991. Y. C. Hsieh was with the Digital Mapping Laboratory, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. he is now with the Mechanical Engineering Department, University of Utah, Salt Lake City, UT 84112. D. M. McKeown is with the Digital Mapping Laboratory, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. F. P. Perlant was with the Digital Mapping Laboratory, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 1.5213. He is now with ISTAR, Sophia Antipolois, France. This work was supported by the U.S. Army Engineer Topographic Laboratories under Contract DACA72-87-C-0001 and by the Avionics Lab, Wright Research and Development Center, Aeronautical Systems Division (AFSC), U. S. Air Force, Wright-Patterson AFB, OH 45433-6543 under Contract F33615-90-C-1465, DARPA Order No. 7597. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the U.S. Army Engineer Topographic Laboratories, the Defense Advanced Research Projects Agency, or of the United States Government. IEEE Log Number 9102685. Relative accuracy of the area-based, feature-based, and merged disparity estimates are provided for several different test scenes.